Search AI Products and News

In today's rapidly developing artificial intelligence landscape, the phenomenon of AI 'hallucinations' has become increasingly frequent, causing significant inconvenience for many companies. Customer service chatbots confidently describe non-existent products, financial AIs fabricate market data, and medical bots provide dangerous medical advice. These issues are no longer mere curiosities; they pose serious risks that are affecting company reputation and profitability. To tackle this challenge, San Francisco-based startup Patronus AI has announced the launch of the world's first self-service platform aimed at real-time detection and prevention.

AI companies like OpenAI are facing pressure from copyright disputes, with The New York Times suing OpenAI for infringement. It is very difficult to remove specific training data from the models, and researchers are exploring a 'forgetting spell.' Researchers have made some progress in testing the forgetting spell using texts, including Harry Potter. The rich language and narrative of Harry Potter are appreciated by AI researchers. Harry Potter appears in the theories of some AI ideologies, such as effective altruism.

The Ma Yi team proposed the EMT framework to evaluate the catastrophic forgetting of fine-tuned Multimodal Large Models (MLLMs). Experiments showed that while fine-tuning MLLMs improves performance on the fine-tuning dataset, it also leads to a decline in performance on other datasets. During the fine-tuning process, MLLMs generate hallucinated text related to the fine-tuning dataset, ignoring the original problem. This research provides a framework and benchmarks for future work, but the design and training techniques of the model still need further optimization. The Ma Yi team systematically assessed the issue of catastrophic forgetting in MLLMs for the first time, balancing different capabilities.